AFRL-AFOSR-VA-TR-2016-0009 Controlled Visual Sensing and Exploration

1 Summary of Research Achievements This project developed analytical and computational tools to design integrated sensor-control systems, where the controller is part of the sensor and designed so as to maximize task-specific information. Within this broad umbrella, we have focused on visual sensors (EO imagery), inertial sensors (accelerometers and gyrometers) and ranging sensors (structured light), and their integration in support of mobility task (exploration) and decisions (detection, localization, recognition, categorization of objects and scenes). The task informs what part of the data-formation process is a nuisance, i.e. it is irrelevant to the task but nevertheless affects the data. Obviously, the resulting sensor-control system depends on the data and it depends on the task. We have focused on tasks that require invariance or co-variance to illumination and to vantage point. Then the control reduces to mobility of the sensor platform, so as to overcome occlusion or scaling limitation in the passive version of the sensor. Therefore, the actuation, control, and sensing systems are collectively considered an active sensor, and algorithms for inference, planning and control can be co-designed so as to achieve maximum uncertainty reduction in the task, or maximum actionable information [11]. 1.1 Modeling the agent The first ingredient to establish an active remote sensor is the ability to move, which requires the ability to localize the sensor platform, or agent, relative to the surrounding environment.